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PREFACE 


In  the  development  of  the  Data  Link  Operational  Experiments 
Program  which  has  an  eventual  goal  of  evaluating  airborne  I/O  de¬ 
vices,  it  has  become  increasingly  evident  that  laboratory  and 
simulator  testing  of  devices  and  concepts  can  provide  useful  in¬ 
formation  which  should  make  possible  a  reduction  in  the  number  of 
devices  and  techniques  requiring  eventual  flight  testing.  The 
present  report  discusses  three  studies  aimed  ct  providing  informa¬ 
tion  of  this  sort. 

The  first  section  describes  evaluation  of  a  visual  display, 
the  WIDCOM,  and  of  a  speech  synthesizer  as  means  for  providing  ATC 
commands  to  experimental  subjects  in  a  GAT-1  simulator.  The  second 
provides  data  on  intelligibility  measurements  of  the  voice  synthesizer. 
The  final  section  describes  a  laboratory  study  to  determine  reaction 
time  differences  when  information  coded  in  various  ways  is  presented 
vo  experimental  subjects. 

The  author  wishes  to  acknowledge  the  contributions  of  numerous 
TSC  personnel  in  equipment  design,  experimental  assistance  and  for 
their  helpful  suggestions  in  clarifying  the  content  of  this  report: 

Mr.  Joseph  Vrable  for  design  of  the  keyboard  interface  for  the 
voice  synthesizer. 

Mr.  Bela  Nagy  for  design  of  the  WIDCOM  visual  display. 

Messrs.  Rudy  Nags  and  John  Gakis  for  the  fabrication  and 
packaging  of  displays  and  interfaces. 

Mr.  Bernard  Patten,  Jr.  for  modifications  to  the  GAT-1  simulator 
and  for  assistance  in  conducting  the  tests  on  the  simulator. 

Mr.  Pruce  Ressler  for  design  of  the  controls  for  the  tests  of 
display  formats. 

Drs .  Donald  Devoe  and  Paul  Abramson  and  Messrs.  Robert  Wiseman, 
Robert  Wisleder,  John  Dumanian  and  Dennis  Collins  for  their  con¬ 
structive  criticisms  of  the  first  draft  of  this  report. 
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1.  SIMULATOR  TESTS 


1.1  INTRODUCTION 

Study  and  planning  for  the  DOT/TSC  Data  Link  program  has 
brought  to  light  the  wealth  of  display  technologies  and  coding 
techniques  which  appear  to  be  suitable  for  presenting  digitally 
transmitted  data.  This  section  reports  on  one  of  a  series  of 
simulator  studies  aimed  at  screening  candidate  devices  so  that 
only  a  limited  number  of  the  most  suitable  will  require  subsequent 
flight  testing. 

The  present  tests  evaluated  two  devices:  a  voice  synthesizer 
and  a  visual  display,  the  WIDCC.4.  The  objectives  of  the  studies 
were  to  determine: 

a.  Could  synthetic  speech  be  generated  with  a  sufficient  ap¬ 
proximation  to  real  time  to  permit  accurate  control  of  a 
flight  path? 

b.  Was  the  intelligibility  of  commands  generated  by  synthetic 
speech  acceptable? 

c.  Could  a  visual  display  of  heading,  altitude  and  speech 
commands  replace  the  present  use  of  voice? 

d.  Were  there  advantages  in  presenting  commands  both 
visually  and  aurally? 

The  voice  synthesizer  used  in  the  experiment  has  been  described 
previously.*  It  provides  solid  state  storage  of  a  vocabulary  of 
one-hundred  and  twenty-eight  words,  stored  as  complete  words  as 
opposed  to  phoneme  storage.  In  its  previous  use  on  the  simulator, 
real  time  message  generation  was  not  possible,  so  that  it  was 
necessary  to  present  messages  on  a  prerecorded  audio  tape  generated 
from  synthetic  speech.  The  inability  to  adjust  timing  or  to  change 
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heading,  altitude  or  speed  commands  with  this  tape  made  accurate 
control  of  desired  aircraft  position  difficult.  During  the  pre¬ 
sent  series  of  runs,  a  keyboard  interface  was  used  which  made  pos¬ 
sible  the  generation  of  commands  in  real  time. 

2 

The  keyboard  interface  was  based  upon  the  TSC  concept  of  a 
ten-key  layout  with  letters  arranged  so  that  successive  striking 
of  the  first  two  letters  of  the  words  in  the  phonetic  alphabet 
uniquely  defined  the  desired  letter,  even  though  multiple  letters 
appear  on  the  key  faces.  For  the  voice  synthesizer,  two  additional 
keys  were  present,  one  denoted  as  and  the  other  as"#".  This 
made  possible  one-hundred  and  forty-four  combinations  and  a  call¬ 
out  of  any  of  the  one-hundred  and  twenty-eight  words  in  the  synthe¬ 
sizer  by  two  keystrokes,  a  majority  of  which  were  mnemonically 
coded.  The  keyboard  interface  also  contained  a  real  time  printer 
and  a  buffer  storage  so  that  messages  could  be  proofread  prior  to 
the  activation  of  the  voice  synthesizer.  The  voice  synthesizer  is 
depicted  in  Figure  1.1,  the  keyboard  interface  in  Figure  1.2  and 
the  code  used  for  vocabulary  callup  is  specified  in  Table  1.1. 

The  output  from  the  voice  synthesizer  was  piped  into  a  GAT-1 
simulator  where  it  could  be  heard  on  a  loudspeaker  or  via  head¬ 
phones  . 

The  WIDCOM,  depicted  in  Figure  1.3,  uses  seven-segment  numerics 
to  give  continuous  information  on  heading,  altitude  and  speed  com¬ 
mands.  Flashing  legend  lights  adjacent  to  the  numeric  readouts 
alert  the  pilot  to  new  commands;  at  the  same  time  a  buzzer  provides 
an  audible  alert.  Pushing  the  flashing  legend  returns  the  light 
to  steady  state  and  silences  the  buzzer.  The  WIDCOM  was  installed 
on  the  upper  left,  corner  of  the  GAT-1  instrument  panel,  as  depicted 
in  Figure  1.4.  The  control  box  for  the  WIDCOM,  depicted  in  Figure 
1.5,  was  installed  next  to  the  GAT-1  X-Y  plotting  board  and  the 
keyboard  interface  for  the  voice  synthesizer. 


2 
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The  McDonnell-Douglas  Voice  Synthesizer 
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Figure  1.2  Keyboard  Interface  for  the 


W  I  DOOM 


WIDCOM  as  Installed 


TABLE  1.1 


KEYSTROKE  CODING  REQUIRED  FOR  ACCESS  TO  THE  VOCABULARY 
OF  THE  SPEECH  SYNTHESIZER 


ABOVE 

G. 

EMERGENCY 

EY 

MARKER 

MZ 

SLOW 

SL 

ACKNOWLEDGE 

## 

FEET 

FH 

MIKE 

MI 

SNOW 

.G 

AFFIRMATIVE 

Y. 

FIVE 

#S 

MILES 

ML 

SOUTH 

S# 

AHEAD 

HA 

FOG 

GF 

MINIMUM 

MH 

SPEED 

AI 

ALFA 

AL 

FOLLOW 

FI 

MINUTES 

MT 

SQUAWK 

.K 

ALTIMETER 

A# 

FOUR  (FOR) 

#  4 

NEGATIVE 

NG 

STOP 

S. 

ALTITUDE 

HI 

FOXTROT 

FO 

NINE 

#9 

TAKEOFF 

T. 

AND 

GH 

FREQUENCY 

FR 

NORTH 

N# 

TANGO 

TA 

APPROACH 

A. 

GOLF 

GO 

NOVEMBER 

NO 

TAXI 

TG 

ASCEND 

GG 

GROUND 

GR 

O'CLOCK 

0. 

TEN 

TE 

AT 

AT 

GUSTY 

GU 

ON 

.N 

THANKS 

HS 

BELOW 

BW 

HEADING 

HE 

ONE 

#1 

THOUSAND 

TH 

BOUND 

H. 

HEAVY 

HY 

OSCAR 

OS 

THREE 

#3 

BRAVO 

BR 

HOLD 

HL 

OUTER 

OE 

TO  (TWO) 

n 

CANCEL 

CA 

HOTEL 

HO 

PAPA 

PA 

TOWER 

.W 

CEILING 

CG 

HUNDRED 

HU 

QUEBEC 

QU 

TRAFFIC 

TR 

CENTER 

CE 

IF  ABLE 

IA 

QUESTION 

.U 

TURBULENCE 

T# 

CHARLIE 

CH 

IFR 

IF 

RADAR 

.D 

TURN 

TU 

CLEARED 

LR 

INCREASE 

I. 

RAIN 

.A 

TWELVE 

TL 

CONTACT 

CO 

INDIA 

IN 

REACH 

RH 

UNIFORM 

UN 

CONTINUE 

GI 

-ING 

IG 

REPEAT 

•  P 

USE 

UZ 

CONTROL 

GL 

INTERMITTENT 

IH 

REPORT 

RE 

VFR 

G# 

DEGREES 

DG 

JULIET 

JU 

REQUEST 

L# 

VICTOR 

VI 

DELAY 

D. 

KILO 

KI 

RESTRICTION 

RG 

VISIBILITY 

.L 

DELTA 

DE 

KNOTS 

KS 

RIGHT 

RR 

WEATHER 

WE 

DEPARTURE 

GA 

LEAVE 

LE 

ROMEO 

RO 

WEST 

W# 

DESCEND 

ZZ 

LEFT 

LL 

RUNWAY 

RW 

WHISKEY 

WH 

DON'T 

DT 

LIGHT 

LT 

RVR 

H# 

WIND 

WI 

EAST 

E# 

LIMA 

LI 

SEVEN 

#7 

X-RAY 

XR 

ECHO 

EC 

LOCATED 

LO 

SIERRA 

SI 

YANKEE 

YA 

EIGHT 

#8 

LOWEST 

LW 

SIX 

#6 

ZERO 

#0 

ELEVEN 

EL 

MAINTAIN 

MA 

SLIPPERY 

LG 

ZULU 

ZU 
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The  location  of  beacons  and  ILS  beams  on  the  GAT-1  simulator 
were  adjusted  to  correspond  with  the  area  surrounding  Boston’s 
Logan  International  Airport,  with  ILS  landings  on  runways  4R  and 
33L . 

It  should  be  pointed  out  that  while  the  WIDCOM  and  voice 
synthesizer  were  evaluated  both  separately  and  concurrently  during 
this  series  of  simulator  runs,  this  should  not  be  considered  a 
comparative  evaluation  of  the  merits  or  deficiencies  of  either 
device  since  they  possess  very  different  total  capabilities. 


1.2  EXPERIMENTAL  PROCEDURE 

In  order  to  develop  four  flight  plan  scenarios  equated  for 
diff ;r.ulty ,  a  piece  of  bent  wire  was  placed  on  the  plotting  board 
map  with  its  end  lined  up  with  Logan's  runway  4R,  and  its  outline 
traced.  The  wire  was  then  rotated  180°  and  the  outline  again 
traced  to  produce  a  mirror  image  of  the  first  flight  path.  The 
same  procedure  was  followed  with  the  same  wire  lined  up  with  runway 
33L.  Altitude  and  speed  command  changes  were  then  interspersed, 
again  inverting  sequences.  The  four  scenarios  then  developed  as: 

Scenario  "J"  Initial  Altitude,  060,  Airspeed  110,  Heading  240 
Descend  to  050 
Airspeed  115 
Right  turn,  heading  330 
Descend  to  040 
Maintain  airspeed  115 
Left  turn,  heading  260 
Descend  to  030 
Airspeed  120 
Right  turn,  heading  350 
Descend  to  020 
Airspeed  130 
Right  turn,  heading  070 
Descend  to  015 
Airspeed  120 
Left  turn,  heading  020 
Cleared,  runway  4  right 


Scenario  "L 


Scenario  "K 


Scenario  "I 


"  Initial  Altitude  020,  Airspeed  115,  Heading  175 
Climb  to  030 
Airspeed  120 
Right  turn,  heading  265 
Climb  to  040 
Airspeed  115 
Left  turn,  heading  195 
Descend  to  030 
Maintain  airspeed  115 
Right  turn,  heading  285 
Descend  to  020 
Airspeed  120 
Right  turn,  heading  005 
Descend  to  015 
Slow  to  110 

Left  turn,  heading  315 
Cleared,  runway  33  left 

"  Initial  Altitude  060,  Airspeed  110,  Heading  095 
Descend  to  050 
Airspeed  115 
Left  turn,  heading  005 
Descend  to  040 
Maintain  airspeed  115 
Right  turn,  heading  075 
Descend  to  030 
Airspeed  120 
Left  turn,  heading  345 
Descend  to  020 
Airspeed  130 
Left  turn,  heading  265 
Descend  to  015 
Airspeed  120 
Right  turn,  heading  315 
Cleared,  runway  33  left 

"  Initial  Altitude  020,  Airspeed  115,  Heading  160 
Climb  to  030 
Airspeed  120 
Left  turn,  heading  070 
Climb  to  040 
Airspeed  115 
Right  turn,  heading  140 
Descend  to  030 
Maintain  airspeed  115 
Left  turn,  heading  050 
Descend  to  020 
Airspeed  120 
Left  turn,  heading  330 
Descend  to  015 
Slow  to  110 

Right  turn,  heading  020 
Cleared,  runway  4  right 
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The  flight  paths  generated  by  these  commands  are  depicted  in 
Figure  1-6.  At  the  scale  used,  commands  occurred  approximately 
every  90  seconds.  The  limitations  of  the  WIDCOM  required  that  the 
final  clearance  to  a  specific  runway  be  delivered  by  voice.  With 
the  voice  synthesizer,  the  commands  were  expanded  from  the  above 
Scenarios,  in  that  headings  included  the  word  "degices",  altitudes 
added  "feet"  and  speeds,  "knots"  for  additional  clarification. 

Eight  licensed  non-professional  pilots,  five  of  them  TSC  person¬ 
nel,  were  used  as  experimental  subjects.  Each  was  given  time  to 
familiarize  himself  with  the  GAT-1  simulator  prior  to  the  experi¬ 
mental  trials.  At  the  start  of  the  experiment  they  were  given  the 
following  written  instructions: 

"You  are  about  to  take  part  in  an  evaluation  of  new  visual  and 
auditory  displays  for  Air  Traffic  Control.  The  visual  display  pre¬ 
sents  continuous  commands  of  heading,  speed  and  altitude;  the 
auditory  display  uses  synthetic  speech  which  can  be  transmitted 
digitally.  The  displays  will  be  used  separately  or  combined  in 
four  different  scenarios,  each  of  which  will  present  a  series  of 
heading,  speed  and  altitude  commands  leading  to  an  eventual  ILS 
approach  to  runway  4  right  or  33  left  at  Logan  cirport. 

"Before  starting  the  formal  experimental  runs,  you  may  take 
as  much  time  as  you  need  to  familiarize  yourself  with  the  simulated 
aircraft.  You  will  also  be  given  a  chance  to  listen  to  the  simu¬ 
lated  speech  and  to  familiarize  yourself  with  the  visual  display. 

"During  the  runs,  altitude  changes  should  be  made  at  500'  per 
minute,  and  turns  at  3°  per  second.  Please  ignore  any  ILS  indica¬ 
tions  until  you  have  been  given  clearance  to  land.  To  save  time 
after  landing  clearance,  you  may  be  given  a  waveoff  so  that  you 
can  start  the  next  run  as  soon  as  possible.  Each  run  will  take 
about  25  minutes . 

IT  IS  IMPORTANT  THAT  YOU  ACKNOWLEDGE  ALL  COMMANDS ,  since  this 
will  be  used  as  a  performance  measure.  Additionally ,  in  the  case 
of  the  auditory-only  display,  you  may  request  a  repeat  of  a  com¬ 
mand  and  may  use  a  scratch-pad  if  you  are  accustomed  to  do  so. 

With  the  auditory  commands,  your  aircraft  ID  will  be  28  Bravo. 

Do  you  have  any  questions?" 

Each  experimental  subject  made  four  simulator  runs  under 
the  following  conditions: 

E:  WIDCOM  without  auditory  alarm. 

F:  WIDCOM  with  buzzer  to  alert  to  new  information. 

G:  WIDCOM  plus  Voice  Synthesizer 

H.  Voice  Synthesizer  alone. 
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To  counterbalance  for  practice  effects,  the  four  experimental 
conditions  (E,  F,  G,  H)  and  the  four  scenarios  (I,  J,  K,  L)  were 
presented  to  the  individual  subjects  using  a  Lc.tin  square  design: 

_ SUBJECTS _ 

1  and  5  2  and  6  3  and  7  4  and  8 

Order  1  El  FL  GK  HJ 

of  2  HK  GJ  FT  EL 

Presentation  3  FJ  EK  HL  GI 

4  GL  HI  EJ  FK 

At  the  completion  of  the  experimental  runs,  the  subjects  were 
asked  to  complete  a  short  written  questionnaire.  All  runs  were 
made  under  IFR  conditions  with  the  cockpit  of  the  simulator  hooded. 
The  time  to  acknowledge  commands  was  measured  with  a  stopwatch. 

In  the  case  of  the  voice  synthesizer,  timing  was  from  the  completion 
of  the  verbal  command  to  the  start  of  the  reply.  With  the  WIDCOM, 
timing  was  from  transmission  of  the  message  to  the  light  up  of  the 
"Acknowledged"  light  on  the  WIDCOM  control  box.  Subjects  normally 
made  two  experimental  runs  in  rapid  succession  with  the  third  and 
fourth  runs  made  at  least  a  day  later. 

1 . 3  EXPERIMENTAL  RESULTS 

The  necessary  differences  in  timing  standards  for  responses 
to  the  WIDCO!:  and  voice  synthesizer,  as  explained  earlier,  makes 
precise  comparisons  impossible.  Ninety-six  percent  of  the  total 
of  four -hundred  eighty  responses  occurred  within  two  to  four 
seconds.  Rather  than  attempt  to  tabulate  mean  and  standard  de¬ 
viations  for  such  data  and  set  a  fixed  value  for  direct  comparison 
with  the  other  sets  of  measurements,  it  seems  more  fruitful  to 
examine  the  number  of  times  that  responses  were  delayed  by  more 
than  five  seconds.  This  approach  was  determined  since  these  are 
the  sort  of  delays  which  could  interfere  seriously  with  overall 
air  traffic  •  mtrol  operations.  Data  for  these  delayed  responses 
are  presented  ;n  Table  1.2;  only  one  of  these  delays  extended 
beyond  10  seconds. 


TABLE  1.2  NUMBER  OF  RESPONSE  TIMES  GREATER  THAN  FIVE  SECONDS 


Voice 

Voice+ 

WIDCOM+ 

WIDCOM 

Subject 

Alone 

WIDCOM 

Buzzer 

NoBuzz 

Total 

1 

3 

0 

3 

0 

6 

2 

0 

0 

0 

0 

0 

3 

0 

3 

1 

2 

6 

4 

0 

0 

1 

0 

1 

S 

0 

4 

0 

0 

4 

6 

0 

1 

0 

1 

2 

7 

0 

0 

0 

1 

1 

8 

0 

0 

0 

0 

0 

Total 

3 

8 

5 

4 

20 

It  is  dangerous  to  generate  firm  conclusions  based  upon  such 
a  small  number  of  data  points.  However,  it  would  appear  that 
(1)  subjects  can  respond  promptly  to  the  majority  of  ATC  commands 
regardless  of  whether  they  are  visual  or  auditory,  and  that  (b) 
certain  subjects  do  not  believe  their  ears  (or  eyes)  and  double 
check  both  modalities  when  possible  before  resp^^ding.  It  is  im¬ 
portant  that  this  be  checked  with  further  simulator  tests  prior  to 
flight  testing  of  data  link  hardware. 

In  Table  1.2,  it  should  also  be  noted  that  only  one  subject 
delayed  his  responses  to  the  "voice  alone"  commands.  Since  the 
visual  display  persisted  until  a  new  command  for  that  particular 
flight  parameter  was  issued,  there  was  no  need  for  subjects  to 
request  a  repeat  of  a  command.  When  commands  were  issued  by 
synthetic  speech  alone,  four  subjects  did  not  request  any  repeats. 
However,  two  asked  for  a  single  repeat  and  two  asked  for  three 
repeats.*  One  of  the  subjects  who  asked  for  three  repeated  com¬ 
mands  later  volunteered  the  information  that  he  was  presently 
undergoing  some  difficulties  in  hearing.  It  thus  appears  that 
the  quality  of  the  synthetic  speech,  while  possibly  marginal,  is 
usable  by  a  majority  of  subjects.  As  a  result  of  the  intelligi¬ 
bility  testing  reported  in  Section  2.  of  this  report,  certain  of 

*It  should  be  noted  that  in  actual  data  link  operation,  repeats 
by  the  controller  would  not  be  necessary,  since  the  message 
would  be  stored  temporarily  aboard  the  aircraft. 
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the  vocabulary  words  are  being  reprogrammed  to  improve  their  clarity. 
This  should  increase  the  acceptability  of  the  synthetic  speech. 


The  results  of  the  questionnaire  administered  to  the  subjects 
at  the  completion  of  their  experimental  runs  can  be  found  in  Ap¬ 
pendix  D.  All  the  subjects  agreed  on  the  clarity  and  desirability 
of  the  WIDCOM,  and  the  WIDCOM  in  conjunction  with  the  voice  synthe¬ 
sizer.  However,  half  the  subjects  found  the  intelligibility  of  the 
synthesizer  alone  to  be  inadequate,  although  they  all  agreed  that 
th.  intelligibility  improved  with  practice.  The  subjects'  suggestions 
for  improvement  on  this  system  are  also  listed  in  Appendix  D. 

1.4  DISCUSSION 

Although  this  series  of  tests  indicated  that  simulation  can 
provide  a  valuable  tool  for  display  evaluation,  it  also  pointed  out 
the  need  for  more  strict  control  of  the  choice  of  hardware  com¬ 
ponents  if  meaningful  evaluations  are  to  be  made.  A  simulator  may 
be  used  to  obtain  information  concerning  the  deficiencies  or  merits 
of  a  given  device,  or  under  the  proper  circumstances,  it  can  pro¬ 
vide  comparative  information  as  to  the  relative  merits  of  two  or 
more  displays.  In  this  latter  instance,  we  should  be  careful  to  do 
comparative  evaluations  of  displays  having  similar  characteristics 
and/or  capabilities. 

The  experiment  reported  herein  provides  a  cogent  example  of 
lack  of  such  experimental  wisdom,  brought  about  by  the  lack  of  a 
variety  of  display  devices,  such  that  no  valid  cross-comparisons 
could  or  should  be  made.  The  visual  display  which  was  evaluated  was 
strictly  limited  as  to  the  types  of  information  which  it  could 
present.  Despite  the  fact  that  the  -uditory  display  could  handle 
a  much  larger  proportion  of  normal  ATC  commands  and  advisories,  it 
could  not  be  exercised  to  this  full  capability  because  of  the 
limitations  of  a  flight  scenario  containing  only  the  heading,  speed 
and  altitude  commands  which  could  be  presented  on  the  visual  display. 
Furthermore,  in  order  to  provide  a  reasonable  workload  in  such  a 
scenario,  changes  of  these  parameters  were  required  so  frequently 
that  no  pilot  subject  could  hope  to  remember  the  continually 
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changing  commands,  and  thus  any  display  which  retained  the  current 
information  on  all  three  of  these  parameters  would  almost  by 
definition,  be  found  to  be  useful  and  desirable. 

On  a  more  positive  note,  the  experiment  demonstrated  that  ac¬ 
curate  control  of  a  flight  path  is  possible  using  a  visual  display, 
and  that  the  information  retention  feature  of  such  a  visual  display 
yields  a  favorable  reaction  from  pilots.  The  comparative  evalua¬ 
tion  of  the  V/IDCOM  with  other  visual  display  devices  is  now  in 
order,  so  as  to  determine  the  comparative  value  of  the  information 
requirements  which  can  be  satisfied  by  a  given  display  device  as 
opposed  to  its  requirements  for  panel  space,  particularly  in  prime 
panel  areas.  Additional  tests  cf  such  visual  displays  in  non-prime 
panel  areas  are  also  in  order,  so  as  to  determine  what  other  posi¬ 
tions  are  allowable  and/or  acceptable,  since  a  device  such  as  the 
WIDCOM  could  certainly  not  justify  its  positioning  in  the  panel 
of  a  commercial  aircraft  as  optimally  as  was  possible  in  the  GAT-1 
simulator. 

The  voice  synthesizer,  since  it  requires  nothing  in  the  way  of 
panel  space,  should  remain  a  candidate  for  eventual  comparative 
evaluation  against  the  best  of  the  visual  displays.  At  the  moment, 
the  intelligibility  of  certain  words  in  its  vocabulary  is  marginal. 

Section  2.  of  this  report  provides  such  intelligibility  data.  Re¬ 
programming  of  these  presently  marginal  words  should  result  in  a 
more  acceptable  device. 

The  inclusion  of  an  out-the -window  task  on  future  simulator 
experiments  should  not  be  overlooked,  since  this  can  provide  ad¬ 
ditional  information  concerning  the  acceptability  of  purely  visual 
ATC  displays  which  require  time-sharing  of  a  single  sensory  modality. 

/ 

1.5  SUMMARY  AND  CONCLUSIONS 

Eight  pilot  subjects  each  made  four  experimental  runs  in  a 
GAT-1  simulator  to  evaluate  synthetic  speech  and  a  prototype  visual 
display  as  means  for  conveying  ATC  command  information.  The  ex¬ 
periment  indicated  (a)  that  synthetic  speech  could  be  generated  with 
a  sufficient  speed  to  permit  accurate  control  of  a  flight  path. 
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(b)  that  the  intelligibility  of  the  synthetic  speech  was  marginal 
for  certain  subjects,  (c)  that  the  visual  display  was  highly  ac¬ 
ceptable  to  pilots,  and  (d)  that  responses  by  pilots  may  be  delayed 
when  information  is  presented  both  visually  and  aurally. 
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2,  INTELLIGIBILITY  TESTING  OF  VOICE  SYNTHESIZER 


2.1  INTRODUCTION 

Report  FAA-RD-72-109*  discussed  methods  for  synthesis  of 
speech,  and  preliminary  testing  of  a  voice  synthesizer  with  a 
75-word  vocabulary  purchased  from  McDonnell-Douglas  Electronics. 

At  about  the  same  time  that  these  earlier  tests  were  being  con¬ 
ducted,  additional  funds  became  available  which  permitted  the 
purchase  of  an  additional  53  words  of  vocabulary,  a  total  of  128 
words.  This  section  of  the  present  report  discusses  intelligibility 
testing  of  the  new  vocabulary  words,  along  with  additional  testing 
of  the  marginal  words  in  the  original  vocabulary  and  of  those  words 
in  the  original  vocabulary  which  appeared  acceptable  on  the  original 
tests  but  which  />ight  be  confused  with  words  in  the  new  vocabulary. 

2 . 2  EXPERIMENTAL  PROCEDURE 

The  output  from  the  voice  synthesizer  was  recorded  on  magnetic 
tape  using  professional  quality  tape  recording  equipment.  This  was 
done  to  avoid  the  possibility  of  any  keying  errors  which  could  re¬ 
sult  in  the  presentation  of  the  wrong  word  during  the  experimental 
session,  and  secondly,  to  permit  ’’playing"  the  experiment  in  a  lab¬ 
oratory  where  the  subjects  sat  in  individual  cubicles  containing 
earphones  connected  to  a  central  tape  player.  Since  the  output  of 
the  voice  synthesizer  has  a  frequency  response  of  telephone  quality, 
it  was  not  felt  that  recording  of  such  output  on  good  quality 
magnetic  tape  could  yield  a  measurable  difference  in  intelligibility. 
Even  though  no  stereo  was  present  in  the  voice  synthesizer  output, 
the  recording  was  made  using  both  channels  of  stereo  equipment  in 
order  to  maintain  a  high  signal-to-noise  ratio. 


*See  Footnote  #1.  (page  1) 
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The  magnetic  tape  also  contained  instructions  to  the  subjects, 
starting  with: 

"This  is  a  test  of  the  intelligibility  of  62  words  in  the 
vocabulary  of  a  solid-state  voice  synthesiser.  The  words 
are  some  of  those  used  in  air  traffic  control.  In  order 
to  give  you  some  familiarity  with  the  way  they  are  pro¬ 
nounced,  we  will  first  present  each  word  three  times  in 
alphabetical  order  while  you  follow  along  with  the  printed 
sheets  which  have  been  supplied  to  you." 

At  the  completion  of  the  vocabulary  presentation,  the  tape 

then  contained  this  announcement: 

"At  this  time  the  experimenter  will  stop  the  tape  while 
the  printed  sheets  are  collected  and  the  scoring  sheets 
are  passed  out.  -  -  -  During  the  scoring  trials,  each 
word  will  be  pronounced  twice.  Please  write  your  name 
at  the  top  of  your  scoring  booklet,  and  we  will  begin. 

Are  you  ready?" 

The  tape  then  contained  five  presentations  of  the  entire 
vocabulary,  each  time  in  a  different  random  sequence,  with  an¬ 
nouncements  interpolated  to  direct  the  subjects  to  start  the 
next  scoring  sheet  at  the  appropriate  time. 

Twenty  female  undergraduates  from  Regis  College  were  used  as 
experimental  subjects.  All  had  passed  complete  audiometer  tests 
and  a  majority  had  participated  in  similar  experiments  previously. 

The  procedure  described  above  differed  somewhat  from  that 
employed  during  the  earlier  tests  of  the  original  75-word  vocabulary, 
in  that  subjects  were  now  required  to  write  down  the  word  rather  than 
to  check  it  off  from  among  multiple  possiblities .  This  was  done  to 
permit  precise  identification  of  the  difference  between  sound-alikes 
such  as  "Mike",  "right"  and  "light,"  since  on  the  earlier  test  each 
of  these  words  had  appeared  only  once  on  a  given  check-off  sheet, 
and  the  remaining  possibilities  could  be  checked  off  by  a  process 
of  elimination. 

2.3  EXPERIMENTAL  RESULTS 

The  specification  for  the  voice  synthesizer  as  purchased  in¬ 
cluded  a  requirement  for  951  recognition  accuracy  for  each  vocabu¬ 
lary.  The  tape  prepared  for  testing  this  contained  three  randomized 
presentations  of  the  62  words  to  be  tested  on  one  side  and  the 
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final  two  on  the  other  side  of  the  tape,  such  that  with  twenty  sub¬ 
jects,  100  responses  would  be  obtained  for  each  word.  However,  it 
was  found  that  when  playing  the  four-track  stereo  recorded  tape  on 
the  monophonic  equipment  at  Regis  College,  there  was  some  cross 
talk  between  opposite  sides  of  the  tape.  Fortunately,  this  occurred 
during  portions  of  only  two  out  of  the  five  presentations  since  it 
was  only  at  the  middle  of  the  length  of  the  tape  that  material  was 
recorded  on  both  sides.  All  data  concerning  presentations  two  and 
five  were  accordingly  disregarded,  and  the  data  presented  below 
is  based  upon  trials  one,  two  and  four  only,  with  a  perfect  score 
being  60  correct  responses. 

Words  which  had  57  or  more  correct  responses,  meeting  the 
95%  recognition  accuracy  criterion  included: 


WORD, 

TIMES 

CORRECT 

WORD 

TIMES 

CORRECT 

ahead 

60 

minimum 

59 

asce’id 

58 

ainutes 

60 

below 

60 

north 

59 

cancel 

60 

outer 

60 

continue 

58 

rain 

57 

degrees 

60 

request 

60 

delay 

60 

restriction 

60 

departure 

60 

RVR 

60 

east 

60 

slippery 

60 

emergency 

60 

slow 

58 

feet 

59 

snow 

58 

follow 

60 

south 

57 

frequency 

60 

stop 

60 

heavy 

59 

takeoff 

60 

if  able 

60 

taxi 

60 

IFR  60 

60 

thanks 

60 

increase 

60 

turbulence 

60 

intermittent 

58 

VFR 

59 

knots 

57 

weather 

60 

light 

59 

west 

60 

lowest 

60 
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Words  deemed  completely  unacceptable  include: 


WORD 

TIMES  CORRECT 

COMMON  WRONG  ANSWERS 

Bravo 

38 

goggle,  throttle,  follow 

fog 

34 

phone,  home,  foam,  fall 

heading 

27 

heavy 

hundred 

46 

upward,  corporate 

ing 

42 

meet,  need 

leave 

19 

home,  plane,  blaze,  laid, 
please,  lane,  lose 

Mike 

21 

might,  light 

repeat 

46 

brigade,  return 

Tango 

51 

to  go,  thank  you,  can  do. 

Twelve  words  remain  in  a  "questionable"  category  and  require 
individual  consideration,  weighing  possible  lack  of  the  subject's 
training  on  the  vocabulary  against  the  consequences  of  failure  in 
recognition  by  pilots  completely  familiar  with  the  vocabulary,  the 
way  that  a  word  is  pronounced  by  the  voice  synthesizer  and  taking 
into  account  the  difficulty  of  recognizing  isolated  words  out  of 
context.  These  words  are: 

a.  "above."  S3  correct  responses,  5  blanks,  heard  as  "upon" 

twice.  Since  its  antonym,  "below"  received  60 
correct  responses ,  there  seems  no  danger  in 
accepting  this  word. 

b.  "acknowledge".  55  correct  responses,  5  blanks.  Not  a  critical 

command  word  and  therefore  should  be  acceptable. 

c.  "and"  52  correct  responses,  3  blanks,  also  heard  as 

"ahead"  and  "end."  Should  be  acceptable  in  its 
present  form. 

d.  "bound"  51  correct  responses,  but  also  heard  as  "found." 

Credit  this  to  lack  of  experience  on  the  part 
of  the  subjects  and  accept  it. 

e.  "control"  53  correct  responses,  5  blanks,  also  heard  as 

"to  go."  There  is  no  serious  hazard  in  accept¬ 
ing  this  word,  since  in  context  it  would  be  used 
regularly  with  "approach"  or  "departure",  etc. 
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f.  "descend" 

g.  "don't" 

h.  "ground" 

i.  "gusty" 

j .  "question" 

k.  "reach" 

l.  "right" 


55  correct  responses,  but  also  heard  as  "ascend". 
This  word  should  definitely  be  reprogrammed, 
even  though  it  is  anticipated  that  "ascend"  will 
be  replaced  by  "climb"  in  a  revised  vocabulary. 

55  correct  responses;  also  heard  as  "built". 
Should  be  accepted. 

52  correct  responses,  3  blanks,  also  heard  as 
"round"  and  "land".  Decision:  acdept. 

56  correct  responses;  also  heard  as  "testing." 
Accept. 

54  correct  responses,  4  blanks.  Also  heard  as 
"less  good".  Decision:  accept. 

56  correct  responses;  also  heard  as  "range". 
Accept . 

52  correct  responses;  also  heard  as  "ride." 
Accept. 


We  are  thus  left  with  a  requirement  for  reprogramming  a  total  of 
10  words  in  the  existing  vocabulary. 


During  the  period  since  the  original  order  was  placed  for  the 
voice  synthesizer,  there  has  been  a  continuing  effort  at  TSC  to 
analyze  the  content  of  air  traffic  control  messages  and  to  select 
the  precise  words  most  useful  in  a  limited  vocabulary.  As  a  re¬ 
sult  of  this  analysis,  certain  changes  in  the  present  vocabulary 
are  anticipated.  Table  2.1  presents  the  anticipated  vocabulary  for 
Data  Link  Flight  Tests  as  it  appears  in  the  recently  issued  specifi 
cation?  New  words,  not  yet  programmed  for  the  voice  synthesizer 
are  identified  by  the  prefix  "N";  old  words  requiring  reprogramming 
are  identified  by  "R", 


- - - - 

Data  Link  Airborne  System  Specification, P66s213-3.0.  July  1972, 
Pages  51-52. 


22 


TABLE  2.1  VOICE  SYNTHESIZER  VOCABULARY  CODES: 
REVISED 


(Words  requiring  reprogramming  indicated  by  "R"; 
new  words  indicated  by  "N.") 

EXTERNAL  EXTERNAL 

ADDRESS  WORD  ADDRESS  WORD 


0 

100 

Oil 

ABOVE 

1 

000 

110 

FOXTROT 

0 

100 

100 

ACKNOWLEDGE 

1 

000 

111 

GOLF 

0 

000 

000 

AFFIRMATIVE 

1 

Oil 

110 

GROUND 

0 

100 

101 

AHEAD 

1 

Oil 

111 

GUSTY 

1 

000 

001 

ALFA 

0 

001 

Oil 

R 

HEADING 

0 

000 

001 

ALTIMETER 

1 

100 

000 

HEAVY 

0 

000 

010 

ALTITUDE 

0 

001 

100 

HOLD 

0 

100 

110 

AND 

1 

001 

000 

HOTEL 

0 

000 

Oil 

APPROACH 

1 

100 

001 

R 

HUNDRED 

0 

101 

111 

N 

Airline* 

1 

100 

010 

IF  ABLE 

0 

000 

101 

AT 

1 

100 

Oil 

IFR 

1 

Oil 

101 

N 

BACKCOURSE 

1 

100 

111 

N 

ILS 

0 

100 

111 

BELOW 

1 

100 

100 

INCREASE 

0 

101 

000 

BOUND 

1 

001 

001 

INDIA 

1 

000 

010 

R 

BRAVO 

1 

100 

110 

INTERMITTENT 

0 

101 

001 

CANCEL 

1 

001 

010 

JULIET 

1 

111 

Oil 

N 

CAN  YOU 

1 

001 

Oil 

KILO 

0 

000 

110 

CEILING 

0 

001 

101 

KNOTS 

0 

000 

111 

CENTER 

1 

110 

000 

N 

LANDING 

1 

000 

Oil 

CHARLIE 

0 

001 

110 

LEFT 

0 

001 

000 

CLEARED 

1 

101 

000 

LIGHT 

0 

000 

100 

N 

CLIMB 

1 

001 

100 

LIMA 

0 

001 

001 

CONTACT 

0 

001 

111 

LOCATED 

0 

101 

010 

CONTINUE 

1 

101 

001 

LOWEST 

0 

101 

Oil 

CONTROL 

0 

010 

000 

MAINTAIN 

0 

101 

100 

DEGREES 

0 

010 

001 

MARKER 

0 

101 

101 

DELAY 

1 

001 

101 

R 

MIKE 

1 

000 

100 

DELTA 

0 

010 

010 

MILES 

0 

101 

110 

DEPARTURE 

1 

101 

010 

MINIMUM 

0 

001 

010 

R 

DESCEND 

1 

101 

Oil 

MINUTES 

0 

111 

110 

EAST 

0 

010 

Oil 

NEGATIVE 

1 

000 

101 

ECHO 

0 

111 

001 

NINE 

0 

111 

000 

EIGHT 

1 

101 

100 

NORTH 

0 

111 

Oil 

ELEVEN 

1 

001 

110 

NOVEMBER 

0 

111 

111 

EMERGENCY 

0 

010 

100 

O'CLOCK 

1 

100 

101 

N 

ESTIMATED 

0 

010 

101 

ON 

1 

000 

000 

FEET 

0 

110 

001 

ONE 

0 

110 

101 

FIVE 

1 

001 

110 

OSCAR 

1 

Oil 

Oil 

R 

FOG 

1 

101 

101 

OUTER 

1 

Oil 

100 

FOLLOW 

1 

010 

000 

PAPA 

0 

110 

100 

FOUR  (FOR) 

1 

010 

001 

QUEBEC 

*This  word  to  be  programmed  for  a  specific  airline  or  aircraft 
(Continued  on  following  page) . 


TABLE  2.1  (CONTINUED) 


EXTERNAL  EXTERNAL 

ADDRESS  WORD  ADDRESS  WORD 


1 

101 

110 

QUESTION 

1 

111 

010 

IAXI 

0 

010 

110 

RADAR 

0 

111 

010 

TEN 

1 

101 

111 

RAIN 

0 

111 

101 

THOUSAND 

1 

111 

000 

N 

REDUCE 

0 

110 

Oil 

THREE 

1 

110 

001 

R 

REPEAT 

0 

Oil 

101 

TOWER 

0 

010 

111 

REPORT 

0 

Oil 

110 

TRAFFIC 

1 

110 

Oil 

REQUEST 

1 

111 

100 

IURBULENCE 

1 

110 

010 

RESTRICTION 

0 

Oil 

111 

TURN 

0 

Oil 

000 

RIGHT 

0 

111 

100 

TWELVE 

1 

010 

010 

ROMEO 

0 

110 

010 

TWO  (TO) 

0 

Oil 

001 

RUNWAY 

1 

010 

101 

UNIFORM 

1 

110 

100 

RVR 

1 

111 

101 

VFR 

0 

110 

111 

SEVEN 

0 

100 

000 

N  VIA 

1 

010 

Oil 

SIERRA 

1 

010 

110 

VICTOR 

0 

110 

110 

SIX 

0 

110 

001 

VISIBILITY 

1 

110 

101 

SLIPPERY 

1 

111 

110 

WEATHER 

0 

Oil 

010 

SLOW 

1 

111 

111 

WEST 

1 

110 

110 

SNOW 

1 

010 

111 

WHISKEY 

1 

110 

111 

SOUTH 

0 

100 

010 

WIND 

0 

Oil 

Oil 

SPEED 

1 

Oil 

000 

X-RAY 

0 

Oil 

100 

SQUAWK 

1 

Oil 

001 

YANKEE 

1 

111 

001 

IAKEOFF 

0 

110 

000 

ZERO 

1 

010 

100 

R 

TANGO 

1 

Oil 

010 

ZULU 

In  this  revised  vocabulary,  the  words  ascend,  don't,  fre¬ 
quency,  -ing,  leave,  reach,  stop,  thanks  and  use  have  been  re¬ 
placed  by  climb,  backcourse,  estimated,  ILS,  landing,  reduce, 
can  you,  via  and  a  blank  which  may  be  used  by  a  specific  airline 
for  its  own  name. 

2.4  SUMMARY  AND  CONCLUSIONS 

Intelligibility  tests  were  run  on  each  of  the  128  words  in 
the  vocabulary  of  a  solid  state  speech  synthesizer,  as  part  of 
the  acceptance  tests  of  the  synthesizer.  Nine  words  were  found 
to  be  completely  unacceptable,  twelve  were  marginal  and  the  balance 
acceptable.  Individual  consideration  was  given  to  the  marginal 
words  on  the  basis  of  possible  lack  of  training  of  the  experimental 
subjects,  the  possible  hazards  inherent  in  incorrect  interpretation 
and  the  difficulty  of  recognizing  individual  words  out  of  context. 
On  this  basis,  only  one  of  the  twelve  marginal  words  requires  re¬ 
programming. 
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3.  COMPARISON  OF  MESSAGE  FORMATS 


3.1  INTRODUCTION 

Information  may  be  encoded  and  presented  in  many  forms.  The 
device  for  displaying  such  information  may,  in  turn,  impose  certain 
restrictions  on  how  the  message  is  encoded.  For  example,  a  device 
capable  of  displaying  only  a  limited  number  of  characters  imposes 
requirements  for  abbreviation  not  needed  with  larger  displays. 

Other  display  devices  may  permit  information  to  be  presented  as  an 
extended  line  of  text  while  others  require  that  the  same  informa¬ 
tion  be  presented  on  several  short  lines.  The  ability  for  an 
observer  to  assimilate  such  information  rapidly  may  vary  as  a 
function  of  such  display  format  and  enforced  coding  limitations. 
This  section  of  this  report  deals  specifically  with  preliminary 
experiments  to  assess  the  effects  of  some  of  these  variables. 

The  development  of  specifications  for  the  airborne  portions 
of  the  Data  Link  Program  within  the  time  constraints  allowed  made 
it  necessary  to  make  certain  somewhat  arbitrary  decisions  as  to 
the  types  of  display  devices  which  would  be  tested. and  the  precise 
manner  in  which  information  would  be  encoded  and  presented  to 
pilots.  Rather  than  to  rely  on  such  arbitrary  decisions  as  a 
basis  for  flight  tests  of  hardware,  it  appeared  that  some  informa¬ 
tion  could  be  obtained  by  relatively  basic  laboratory  experiments, 
and  the  ffirst  of  such  experiments  is  the  subject  of  this  section 
of  this  report. 

3.2  OBJECTIVES 

This  experiment  attempted  to  obtain  information  on  such 
display  variables  as: 

a.  Type  font.  Are  there  differences  in  speed  or  accuracy 
or  reaction  when  ATC  information  is  presented  as  dot 
matrix  characters,  stencil  type  characters  or  as  portions 
of  16-segment  arrays? 
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b.  Differences  in  readability  between  an  extended  linear  \ 

display  and  the  same  information  presented  on  three  | 

short  lines. 

c.  Arrows  versus  words.  Are  arrows  denoting  ’’climb", 

"descend",  "right",  and  "left"  recognized  more  rapidly 
than  the  words  or  their  abbreviations? 

d.  Differences  in  reaction  time  and  error  rate  between  the 
presentation  of  purely  qualitative  information  and  of 

information  prdviding  quantitative  values  of  parameters. 

• 

e.  Differences  between  presenting  a  new  command  by  itself 
and  presenting  the  new  command  while  maintaining  a 
"scratchpad"  of  the  old  values  of  other  parameters. 

f.  When  both  new  and  old  material  is  presented  in  a  complex 
message,  is  it  necessary  to  present  the  new  material  at 
the  beginning  of  the  message,  or  can  material  be  presented 
in  a  fixed  sequence  of  parameters,  with  means  for  delineat¬ 
ing  which  is  the  new  information? 

These  and  other  parameters  were  explored  in  the  following 
experiment. 

3.3  EXPERIMENTAL  CONDITIONS 

Eight  formats  for  commands  were  generated  under  each  of  the 
six  broad  categories  "climb",  "descend",  "turn  right",  "turn  left", 

"tune  your  transceiver"  and  "this  is  a  message  requiring  acknowledge¬ 
ment."*  Artwork  was  prepared  for  each  of  these  48  messages  using 
three  type. .fonts;  a  5  x  7  dot  matrix,  characters  simulating  those 
from  a  16-segment  array,  and  characters  simulating  stencils  such  as 
might  appear  on  the  face  of  a  Charactron  CR".  The  artwork  was 
then  photographed  to  provide  a  total  of  144  double  frame  35  mm. 
high-contrast  negative  slides  (white  characters  on  a  black  background). 

The  slides  were  then  mounted  in  2"  by  2"  slide  carriers, 
randomized  in  order  and  distributed  evenly  between  two  slide  trays. 

*The  precise  formats  for  each  of  the  144  slides  are  reproduced  along 
with  the  raw  data  in  Appendix  A. 
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Each  slide  additionally  contained  a  clear  spot  in  the  upper 
right  hand  corner  to  permit  the  activation  of  a  photocell  when  the 
slide  was  presented  on  a  screen. 

Equipment  for  the  experiment,  other  than  the  slides,  consisted 
of  a  3S  mm.  slide  projector,  a  projection  screen  with  affixed  photo¬ 
cell,  a  response  box  for  subjects,  with  six  pushbuttons  to  permit 
the  subjects  to  indicate  their  interpretation  of  the  message  in 
terms  of  the  appropriate  response,  a  series  of  numbered  lights  to 
permit  the  experimenter  to  ascertain  the  correctness  of  the  subjects' 
response,  and  an  interval  timer  calibrated  in  hundredths  of  a  second 
which  automatically  measured  the  time  from  the  appearance  of  the 
slide  to  the  subjects'  responses. 

Ten  engineers  and  scientists  from  TSC  were  used  as  experi¬ 
mental  subjects.  While  four  of  these  were  members  of  the  Data 
Link  team,  none  had  any  precise  knowledge  concerning  the  variables 
which  were  being  explored. 

All  subjects  were  handed  a  typewritten  sheet  containing  the 
instructions : 

"IT  IS  IMPORTANT  THAT  YOU  READ  THESE  INSTRUCTIONS  CAREFULLY, 
TAKING  ALL  OF  THE  TIME  THAT  YOU  WISH,  SINCE  YOUR  PERFORMANCE 
ON  THIS  TEST  WILL  DEPEND  TO  A  LARGE  EXTENT  ON  HOW  WELL  YOU 
HAVE  ABSORBED  THE  INFORMATION  ON  CODING  AND  ABBREVIATIONS. 

This  is  a  study  to  determine  how  best  to  present  some  of  the 
commands  which  will  be  issued  to  pilots  via  Digital  Data  Link 
during  forthcoming  flight  tests.  In  this  experiment,  slides 
will  be  presented  on  a  screen  and  you  will  be  asked  to  re¬ 
spond  as  rapidly  and  accurately  as  possible  to  the  various 
types  of  command. 

Your  control  box  has  six  buttons.  The  four  central  buttons, 
arranged  in  a  diamond-shaped  pattern,  Represent  your  aircraft 
controls  for  up,  down,  right  and  left.  The  button  on  the 
extreme  left  represents  your  control  of  the  frequency  of  your 
radio  transceiver.  The  button  on  the  extreme  right  is  used 
to  acknowledge  all  other  commands  or  advisories.  You  are  thus 
required  to  interpret  the  message  before  making  a  response. 

You  will  be  scored  both  for  the  accuracy  and  speed  of  your 
response,  although  accuracy  is  the  preferred  criterion. 
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Various  types  of  abbreviations  will  be  used  at  the  start 
of  messages: 

A  =  ALT  =  ALTITUDE,  modified  by  up,  down,  climb,  descend 
or  appropriate  arrows. 

C  =  CONTACT.  This  indicates  a  command  to  change  radio 
frequency. 

H  =  HDG  =  HEADING,  modified  by  R,  Right,  L,  Left  or 
appropriate  a  rows . 

S  =  SPD  =  Speed 

T  =  TURN,  modified  by  R,  RIGHT,  L,  LEFT  or  appropriate 
arrows . 

When  single-letter  abbreviations  are  used,  the  ones  listed 
above  always  appear  first,  but  may  be  followed  in  the  case 
of  a  radio  frequency  command  by  a  second  single-letter  ab¬ 
breviation  to  indicate  a  specific  controller  with  the  fol¬ 
lowing  specific  interpretations: 

CT  =  CONTACT  TOWER 

CA  =  CONTACT  APPROACH  CONTROL  (Note  that  the  "A"  in  second 
position  stands  for  "Approach",  not  "Altitude". 

CG  =  CONTACT  GROUND  CONTROL 

On  some  slides,  you  may  see  multiple  categories  of  infor¬ 
mation.  In  this  case,  the  new  information  to  which  you 
should  respond  is  set  off  by  astericks,  e.g.,  HDG  230 
*ALT+.160*  SPD  220.  In  the  example  listed,  the  appropriate 
response  is,  of  course,  to  press  the  "climb"  button. 

Examples  of  commands  which  require  the  use  of  the  "Acknowledge" 
button  are: 

MAINTAIN  ALTITUDE 
HOLD  SPEED 

SQUAWK  (This  supplies  a  setting  for  your  transponder)  . 

RADAR  CONTACT 
CLEARED  FOR  TAKEOFF 
TRAFFIC  12  O'CLOCK  2  MILES 

You  will  have  only  one  chance  to  respond  to  each  slide.  Do 
you  have  any  questions?" 

As  further  training,  the  subjects  were  then  handed  the  re¬ 
sponse  box  layout  depicted  in  Figure  3.1,  demonstrating  the  possible 
coding  for  each  of  the  six  numbered  control  buttons,  and  were  urged 
to  check  out  possible  finger  placement  on  the  actual  control  box  to 
facilitate  their  responses. 

Subjects  were  run  individually,  and  the  average  total  time  per 
subject  was  approximately  25  minutes.  In  order  to  counterbalance 
the  effect  of  practice,  slide  tray  2  was  presented  first  to  the 
last  five  subjects. 
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Since  this  was  a  test  of  the  recognizability  of  the  infor¬ 
mation  and  not  of  visual  acuity,  the  characters  of  the  messages 
were  projected  at  a  height  of  approximately  1/2"  on  the  screen, 
and  the  subjects  viewed  them  from  a  distance  of  approximately  30". 

During  the  experiment,  the  subjects  usually  volunteered  the 
information  that  they  were  aware  when  they  had  made  an  error  in 
their  response.  When  this  information  was  not  volunteered,  the 
conductor  of  the  experiment  pointed  out  the  error  to  facilitate 
performance  on  the  remaining  slides.  The  raw  data  from  the 
experiment,  organized  by  subject  and  slide  categories  are  pre¬ 
sented  in  Appendix  A.  The  actual  sequence  in  which  the  slides 
were  presented  to  the  subjects  is  indicated  in  Appendix  B. 

3 . 4  EXPERIMENTAL  RESULTS 

Experimental  design  must  always  represent  a  compromise  between 
limitations  on  the  number  of  variables  to  be  explored  and  the  time 
needed  to  obtain  the  required  number  of  data  points  to  permit 
meaningful  comparisons.  Where  a  completely  balanced  experimental 
design  is  possible,  analysis  of  variance*  provides  a  powerful  tool 
for  determining  relationships  among  variables,  but  with  an  in¬ 
crease  in  the  number  of  required  data  points  if  all  combinations 
of  all  conditions  are  to  be  tested.  Thus  an  increase  is  required 
in  the  number  of  experimental  man-hours.  Conversely,  simple  t-tests 
of  differences  among  multiple  mean  values  are  prone  to  misinterpre¬ 
tation  unless  a  requirement  for  a  high  level  of  statistical 
significance  is  enforced. 

A  reasonable  alternative  is  to  use  analysis  of  variance  of 
those  data  blocks  which  are  complete  to  generate  an  error  term 
suitable  foi  use  with  multiple  t-tests,  and  this  technique  has 
been  applied  in  the  reduction  of  the  present  data. 


*A  brief  summary  of  statistical  terminology  is  included  in 
Appendix  C. 


:> 
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From  the  tabulated  raw  data  reproduced  in  Appendix  A,  the 
means  of  non-overlapping  variables  have  been  selected  and  are 
reproduced  in  Table  3.1* 


TABLE  3.1  MEANS  FOR  NON-OVERLAPPING  VARIABLES 
Disjunctive  reaction  time  in  seconds. 


MESSAGE 

CHARACTER  FONT 

TYPE 

ST 

SEG 

SINGLE  WORD 

0.96 

0.84 

0.99 

ARROWS  ONLY 

0.81 

0.76 

0.79 

WORDS  +  NUMBERS 

1.06 

1.00 

1.11 

ARROW  +  NUMBERS 

0.96 

1.08 

0.95 

3- LINE:  WORDS 

0.96 

1.10 

1.02 

3- LINE:  ARROWS 

0.96 

0.83 

1.03 

1-LINE:  WORDS 

1.52 

1.66 

1.80 

1-LINE:  ARROWS 

1.51 

1.74 

1.70 

*In  generating  the  tables  for  this  subsection,  data  points  were  el¬ 
iminated  in  those  few  cases  where  a  single  subject  recorded  a 
response  time  for  a  particular  slide  which  was  more  than  twice 
the  response  time  for  any  other  subject,  since  such  data  pc.nts 
would  probably  indicate  momentary  inattention  by  that  subject. 
Also,  in  these  tables,  means  for  subjects  for  any  particular 
slide  were  deleted  when  there  were  fewer  than  six  measurable  re¬ 
sponses  for  the  ten  subjects.  The  numbers  in  the  tables  ac¬ 
cordingly  represent  means  for  at  least  six  subjects  for  the 
selected  slides  and  conditions  to  be  compared,  with  times  recorded 
in  seconds  and  hundredths  of  a  second.  In  the  majority  of  these 
cases,  three  such  numbers  appear  for  a  given  message,  these  re¬ 
presenting  the  response  times  for  the  same  message  when  presented 
respectively  with  dot  matrix  (DM) ,  stencil  (ST)  and  16-segment 
(SEG)  fonts. 

4* 

Disjunctive  reaction  time  is  the  time  required  for  a  response  to 
a  stimulus  which  requires  selection  of  the  appropriate  action 
from  multiple  possible  choices. 
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Analysis  of  variance  for  these  data  was  calculated,  and  is  sum¬ 
marized  in  Table  3.2. 


TABLE  3.2 

ANALYSIS  OF  VARIANCE  FOR  THE 

DATA  OF 

TABLE  3.1 

SOURCE 

SUM  OF 

DEGREES  OF 

MEAN 

F-RATIO 

SQUARES 

FREEDOM 

SQUARE 

Message  Type 

233.73 

7 

33.39 

35. 9 

Font 

2.67 

2. 

1.34 

1.44 

Interaction 

13.03 

14 

0.93 

Total 

246.76 

23 

Using  the  error  term  thus  calculated  and  the  values  of  the 
t-distribution  for  14  degrees  of  freedom,  the  required  differences 
between  means  for  various  levels  of  significance  may  be  calculated 
using  the  formula: 


where  Mn  is  the  mean  difference  between  measures  being  compared, 

2  u 

s  is  the  variance,  N  is  the  number  of  data  points  being  compared 
and  t  is  the  value  obtained  from  tables  of  the  t-distribution.  The 
results  of  such  calculations  for  various  levels  of  statistical 
significance  are  presented  in  Table  3.3. 

TABLE  3.3  MEAN  DIFFERENCES  REQUIRED  FOR  MESSAGE  TYPE/FONTS  FOR 
VARIOUS  SIGNIFICANCE  LEVELS 

Required  differences  in  seconds 


SIGNIFICANCE  LEVEL 

0.1 

0.5 

0.02 

0.01 

0.001 

t-distribution  £4>r 
df  =  14 

1.761 

2.145 

2.624 

2.977 

4.140 

MEAN 

MESSAGE 

TYPE 

FONTS 

0.085 

0.103 

0.127 

0.147 

0.200 

DIFFERENCE 

MD 

0.139 

0.170 

0.207 

0.234 

0.326 
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A  prime  consideration  in  this  experiment  was  to  determine  the 
differences,  if  any,  between  responses  to  the  type  fonts,  since 
ruling  out  of  significant  differences  here  could  greatly  simplify 
the  preparation  of  artwork  for  any  subsequent  experiments.  The 
analysis  of  variance  of  Table  3.2  indicated  lack  of  significance 
of  fonts  a-?  an  experimental  variable.  This  is  substantiated  by 
the  data  of  Table  3.4,  where  response  times  to  the  several  types 
of  commands  are  compared  for  the  three  types  fonts  used. 

However,  this  generalization  is  not  valid  when  directional 
arrows  are  included  within  textual  material.  With  a  16-segment 
array,  it  is  possible  only  to  generate  arrows  half  the  size  of 
other  characters,  as  indicated  below.  Locating  and  interpreting 
these  half-size  arrows  within  textual  material  increases  reaction 
time  appreciably  over* that  for  the  same  message  displayed  in  dot 
matrix  or  stencil  type  characters. 


Data  for  "buried"  arrows  are  presented  in  Table  3.5  The 
differences  here  would  probably  be  greater  if  better  quality  ar¬ 
rows  had  been  used  in  the  stencil  type  font.  To  facilitate  the 
artwork  preparation,  the  arrows  were  made  by  adding  a  line  to  the 
letter  "V"  and  placing  the  letter  in  the  appropriate  orientation. 

It  thus  was  less  noticable  in  alphanumeric  material  than  would  be 

an  arrow  such  as  - ^  or  .  .  .  The  differences 

between  lC-segment  and  dot  matrix  fonts  were  significant  at  the 
0.01  level. 

Arrows  by  themselves  are  better  than  single  words  for  IPC-type 
commands  as  indicated  in  Table  3.6.  Differences  are  significant 
at  the  0.02  level. 
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TABLE  3.4  COMPARISON  OF  TYPE  FONTS 


Mess  age  type  means  TgfUPfWX'C'  mean's  for  a  maximum  o£  80  measure¬ 
ments.  Disjunctive  reaction  times  in  seconds. 

. . . 


TYPE  FONT 

MESSAGE  TYPE 

DM 

ST 

SEG 

RADIO 

l.SS 

1.49 

1.63 

LEFT  TURNS 

1.21 

1.08 

1.18 

CLIMB  COMMANDS 

1.06 

1.05 

1.19 

DESCEND  COMMANDS 

0.99 

1.12 

1.15 

RIGHT  TURNS 

1.29 

1.14 

1.12 

ACKNOWLEDGMENTS 

1.38 

1.37 

1.56 

MEAN 

1.24 

1.21 

1.31 

TABLE  3.5  TYPE  FONT  COMPARISON  FOR  "BURIED”  ARROWS 
Disjunctive  reaction  time  in  seconds. 


DOT  MATRIX 

STENCIL 

SEGMENTED 

Slide 

Time 

Slide 

Time 

Slide 

Time 

10 

0.96 

58 

0.98 

106 

1.23 

13 

1.16 

61 

1.25 

109 

1.35 

19 

1.22 

67 

1.71 

115 

2.30 

21 

1.56 

69 

1.14 

117 

1.52 

27 

1.20 

75 

1.87 

123 

1.67 

29 

1.28 

77 

1.52 

125 

1.32 

34 

0.93 

82 

1.01 

130 

0.99 

37 

1.58 

85 

0.97 

133 

1.41 

MEAN 

1.24 

1.31 

1.47 
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TABLE  3.6  SINGLE  WORDS  VERSUS  ARROWS 


Disjunctive  reaction  time  in  seconds. 


SINGLE 

WORD 

ARROWS 

COMMANDS 

ONLY 

LEFT 

DM 

1.09 

-«-DM 

0.84 

ST 

0.82 

ST 

0.76 

SEG 

1.04 

SEG 

0.71 

CLIMB 

DM 

0.86 

+  +  DM 

0.74 

ST 

0.78 

ST 

8*71 

SEG 

1.00 

SEG 

0.83 

DESCEND 

DM 

0.83 

44DM 

0.79 

ST 

0.95 

ST 

0.73 

SEG 

0.93 

SEG 

0.90 

RIGHT 

DM 

1.05 

-♦■-►DM 

0.87 

ST 

0.81 

ST 

0.82 

SEG 

0.97 

SEG 

0.72 

MEALS 

0.93 

0.79 

On  the  other  hand,  the  differences  between  ireans  between  words 
and  arrows  when  numerical  values  are  added  to  the  display,  as  de¬ 
picted  in  Table  3.7  are  non-significant.  Arrows  again  show  non¬ 
significant  differences  from  words  in  3- line  messages  indicating 
a  single  command,  as  indicated  in  Table  3.8. 

The  comparison  between  the  use  of  arrows  '  one  (an  IPC  command) 
and  arrows  along  with  numerical  values,  depicted  in  Table  3.9  again 
indicates  the  superiority  of  the  simple  IPC  command.  Since  the 
differences  here  are  significant  at  the  0.02  level,  consideration 
should  accordingly  be  given  to  the  use  of  arrows  alone  for  emergency 
maneuvers,  with  numerical  data  added  only  after  the  maneuver  has 
begun. 

The  differences  between  text  alone  versus  text  with  numerical 
values  for  simple  commands  such  as  "climb"  are  less  striking,  as 
indicated  in  Table  3.10,  but  are  significant  at  the  0.05  level. 
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TABLE  3.7  WORDS  VERSUS  ARROWS  WITH  NUMERICAL  VALUES  ADDED 


Disjunctive  reaction  time  in  seconds. 


MESSAGE 

TIME 

MESSAGE 

TIME 

CLMB210 

DM 

1.11 

A+120 

DM  1.00 

ST 

1.10 

ST  1.02 

SEG 

1.23 

SEG  0.88 

DOWN  120 

DM 

1.01 

A+120 

DM  0.91 

ST 

0.99 

ST  1.14 

SEG 

0.99 

SEG  1.02 

MEAN 

1.06 

1.00 

TABLE  3.8  3 -LINE  MESSAGES:  WORDS  VERSUS  ARROWS 


Disjunctive  reaction  time  in  seconds 


MESSAGE 

TIME 

MESSAGE 

TIME 

TURN 

DM  0.83 

TURN 

DM  0.94 

LEFT 

ST  0.97 

ST  --- 

180 

SEG  0.89 

290 

SEG  1.13 

TURN 

DM  1.08 

TURN 

DM  0.98 

RIGHT 

ST  1.22 

-*-* 

ST  0.83 

090 

SEG  1.15 

110 

SEG  0.92 

MEAN 


0.79 


0.96 


TABLE  3.9  ARROWS  ALONE  VERSUS  ARROWS  WITH  NUMERICAL  VALUES  ADDED 


TABLE 


MEAN 


Disjunctive  reaction  time  in  seconds. 


ARROWS 

ALONE 

ARROWS 

+  NUMBERS 

Slide  No. 

Time 

Slide 

No.  Time 

IS 

0.84 

16 

0.94 

63 

0.76 

64 

1.40 

111 

0.71 

112 

1.13 

23 

0.74 

20 

0.87 

.  71 

0.71 

68 

0.95 

'  119 

0.83 

116 

0.94 

31 

0.79 

28 

0.93 

79 

0.73 

76 

1.00 

127 

0.90 

124 

1.23 

39 

0.87 

40 

0.98 

87 

0.82 

88 

0.83 

13S 

0.72 

136 

0.92 

MEAN 

0.79 

0.92 

.10  COMPARISON  OF  MESSAGES  HAVING  TEXT  WITH  AND  WITHOUT 
NUMERICAL  VALUES 


Disjunctive  reaction  time  in  seconds 


TEXT: 

NO  NUMBERS 

TEXT  WITH 

NUMBERS 

Slide 

Time 

Slide 

Time 

14 

1.09 

12 

0.83 

62 

0.82 

60 

0.97 

110 

1.04 

108 

0.89 

22 

0.86 

18 

1.11 

70 

0.78 

66 

1.10 

118 

1.00 

114 

1.23 

30 

0.83 

26 

1.01 

78 

0.95 

74 

0.90 

126 

0.92 

122 

0.99 

38 

1.05 

36 

1.08 

86 

0.81 

84 

1.22 

134 

0,97 

132 

1.15 

C . 93  1.P4 
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In  quite  a  different  vein,  Table  3.11  depicts  the  differences 
between  the  use  of  a  single  long  line  and  of  three  short  lines  for 
presenting  three  parameters  of  data.  The  single  line  format  is 
typical  of  the  Burroughs  Self  Scan  Display,  whereas  the  3-line 
format  would  typically  involve  the  use  of  individual  Light  Emit¬ 
ting  Diode  (LED)  characters.  The  differences  in  favor  of  the 
3-line  display  are  significant  at  the  0.001  level. 

We  have  also  been  concerned  with  needs  for  putting  new  in¬ 
formation  on  a  parameter  first  or  at  the  top  when  a  scratchpad  is 
maintained  on  values  of  other  parameters.  The  data  of  Tables  3.12 
and  3.13  indicate  that  reaction  time  is  appreciably  shorter  when 
the  new  information  is  in  front  or  on  top,  both  significant  at  the 
0.05  level.  Whether  or  not  these  differences  are  sufficiently 
great  as  to  justify  the  extra  computer  programming  required  to 
place  new  information  at  the  front  or  top  and  disp  ’ce  older  in¬ 
formation  remains  a  question  to  be  resolved. 

TABLE  3.11  LINEAR  VERSUS  THREE-LINE  PRESENTATION  OF  THREE  PARAMETERS 


LINEAR 

THREE-LINE 

Slide 

Time 

Slide 

Time 

11 

1.93 

13 

1.16 

59 

— 

61 

1.25 

107 

1 . 35 

109 

1.35 

19 

1.22 

21 

1.56 

67 

1.71 

69 

1.19 

115 

2.30 

117 

1.52 

27 

1.21 

29 

1.28 

75 

1.87 

77 

— 

123 

1.67 

125 

1.32 

35 

1.67 

37 

1.58 

83 

1.65 

85 

0.97 

131 

1.49 

133 

L_41 

MEAN 

1.64 

1.33 
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TABLE  3.12  COMPARISON  OF  POSITION  OF  NEW  INFORMATION  WITH  SINGLE- 
LINE  FORMAT 


Disjunctive  reaction  time  in  seconds. 


NEW  INFORMATION 

"BURIED" 

NEW  INFORMATION 

IN  FRONT 

Slide 

Time 

Slide 

Time 

4 

1.57 

7 

1.61 

52 

2.10 

55 

1.97 

100 

1.96 

103 

1.28 

19 

1.22 

11 

1.93 

67 

1.71 

59 

— 

115 

2.30 

101 

1.35 

27 

1.21 

35 

1.67 

75 

1.87 

83 

1.65 

123 

1.67 

131 

1.49 

MEAN 

1.73 

1.62 

TABLE  3.13  COMPARISON  OF  POSITION  OF  NEW  INFORMATION  WITH  THREE- 
LINE  FORMAT 

Disjunctive  reaction  time  in  seconds 


NEW  INFORMATION 

"BURIED" 

NEW  INFORMATION 

ON  TOP 

Slide 

Time 

Slide 

Time 

21 

1.56 

13 

1.16 

69 

1.19 

61 

1.25 

117 

1.52 

109 

1.35 

29 

1.28 

37 

1.58 

77 

1.52 

85 

0.97 

125 

1.32 

133 

1.41 

MEAN 

1.40 

1.29 
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Table  3.14  depicts  the  most  striking  difference  found  in 
all  of  the  data,  involving  an  unfortunate  choice  of  symbology 
to  fill  a  7-window  display.  Specifically,  the  symbology  HDGLXXX 
and  HDGR,  where  X  represents  a  digit,  should  be  avoided,  whereas 
HDG+XXX  and  HDG+XXX  are  reacted  to  rapidly  and  accurately. 


TABLE  3.14  ARROW  VERSUS  "BURIED"  "L"  OR  "R". 


MESSAGE 

TIME 

MESSAGE 

TIME 

HDGL210 

DM 

1.96 

HDG«-230 

DM 

0.96 

ST 

1.05 

ST 

0.98 

SEG 

1.74 

SEG 

1.23 

HDGR110 

DM 

2.19 

HDG-+-120 

DM 

0.93 

ST 

1.77 

ST 

1.01 

SEG 

1.32 

SEG 

0.99 

MEAN 

1.67 

1.02 

Throughout  the  experiment,  the  subjects  maintained  a  high 
degree  of  accuracy  in  their  choice  of  the  correct  pushbutton 
for  a  particular  response,  with  a  mean  error  rate  of  approximately 
2.5  percent.  Error  data  are  presented  in  Table  3.15,  but  the 
total  number  of  errors  is  too  small  to  permit  drawing  conclusions. 

The  final  table,  Table  3.16,  presents  data  for  the  individual 
subjects  in  terms  of  mean  reaction  times  for  the  six  message 
categories.  The  distribution  of  reaction  times  appears  normal. 
Differences  in  reaction  time  among  messages  categories  should 
be  ignored  since  we  have  no  means  of  determining  the  time  dif¬ 
ferences  required  for  finger  movement  to  the  appropriate  response 
button  for  a  particular  slide. 

Finally,  there  appears  to  be  no  correlation  between  the  re¬ 
sponse  time  data  and  the  number  of  errors  made  by  individual 
subjects,  as  presented  in  Tables  3.15  and  3.16 
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TABLE  3.15 


THE  NUMBER  OF  ERRORS  TO  ONE -HUNDRED  FORTY -FOUR  VISUAL 
AIRCRAFT  COMMAND  MESSAGES  FOR  A  SAMPLE  OF  TEN  SUBJECTS 
AND  SIX  CONTROL  COMMANDS 


BUTTON 

CONTACT 

RADIO 

1 

TURN 

ISA  T 
•> 

CLIMB 

3 

nnscr.ND 

4 

TURN 

RIGHT 

S 

ACKNOW¬ 

LEDGE 

6 

TOTAL 

PER 

SUBJECT 

SUBJICT 

m 

m 

m 

□ 

m 

2 

m 

§2 

m 

|U| 

IB 

m 

see 

SI 

i 

l 

1 

■ 

1 

HBi 

i 

»n 

S3 

l 

n 

n 

1 

S4 

i 

H 

2 

1 

n 

Kl 

3 

5  5 

iu 

■u 

■u 

■i 

■■ 

H 

n 

1 

Hi 

■■ 

H 

1 

n 

1 

■I 

■i 

1 

tn 

H 

w 

B 

1 

n 

n 

1 

S8 

E_ 

n 

S 

1 

n 

n 

1 

■I 

n 

2 

\mm 

_ 

1 

n 

■re 

n 

1 

Kl 

X 

sio 

!H 

— 

■ 

m 

_ 

_ 

i 

D 

^■1 

5 

u 

2 

TOTAL  PLR  BUTTON 

in 

B 

H 

m 

H 

B 

a 

B 

m 

m 

B 

B 

B 

B 

B 

n 

B 

HI 

IB 

B 

17 

TABLE  3.16  MEAN  RESPONSE  TO  ONE-HUNDRED  FORTY-FOUR  VISUAL  AIRCRAFT 
COMMAND  MESSAGES  FOR  A  SAMPLE  OF  TEN  SUBJECTS  AND  SIX 
CONTROL  COMMANDS  VERSUS  RESPONSE  TIME  IN  SECONDS 


\1 BUTTON 
SUBJECT'"'-^ 

CONTACT 

RADIO 

1 

CLIMB 

3 

DESCEND 

4 

mB 

'yWljyB 

ACKNOW¬ 

LEDGE 

6 

MEAN 

PER 

SUBJECT 

SI 

1.19 

0.86 

0.72 

0.85 

0.81 

0.95 

0.90 

S2 

“05 

BUM 

ma 

1.14 

BOKSBI 

1.S4 

BEKH^BBI 

S3 

1 .64 

BM 

IBM 

HUH 

1711 

■H«l 

HBWHHBB 

S4 

1.62  ’’ 

BUI 

IHH 1 

ilEIBi 

1.17 

■BIH 

BnR^H 

SS 

in  W 

nm 

■(HI 

1.17 

1  .36 

m FXTBBB 

S6 

1.S8 

1.20 

EKil 

1.13 

1.44 

mnH 

S7 

1 . 56 

1.13 

1.22 

1.09 

1.18 

.  -1  •  7  0 _ 1 

1  1.31  I 

S8 

1.45 

wntm 

■HBi 

1.15 

1.09 

imi 

IBHUBBB 

S9 

1.72 

UKUB 

ua 

BDKlLBBI 

mmsm 

■I4« 

ibbmlbbb 

SIO 

IBM 

BR> 

IBM 

■DB 

1.33 

■win 

IBBBPBBB 

MEAN  PER  BUTTON 

1 . 56 

1.14 

1.09 

1.08 

1.19 

1.43 

1.25 

3.5  SUMMARY  AND  CONCLUSIONS 

A  series  of  one  hundred  and  forty-four  slides  was  prepared 
representing  ATC  messages  in  six  general  categories,  with  varia¬ 
tions  in  message  format,  coding  and  type  fonts.  The  slides  were 
presented  individually  to  ten  subjects  and  disjunctive  reaction 
times  measured.  The  results  of  the  experiment  indicate  that: 

a.  There  are  no  differences  in  reaction  time  resulting  from 
the  use  of  different  type  fonts  except  when  arrows  as 
symbology  are  "buried"  within  the  text.  Here,  the  use 
of  16-segment  font  should  be  avoided. 

b.  Arrows  aie  better  than  words  or  abbreviations  for  simple 
IPC  commands  or  for  short  messages  containing  numerical 
values  of  a  parameter. 

c.  For  an  emergency  situation,  only  arrows  should  be  pre¬ 
sented,  followed  later,  if  necessary,  by  a  numerical 
value. 

d.  Multiple  commands  are  preferably  presented  on  three 
short  lines  rather  than  one  extended  line. 

e.  "New"  information  should  preferably  be  presented  at 
the  top  or  left  of  a  display  which  maintains  a  scratch¬ 
pad  of  the  previous  values  of  other  parameters. 

f.  The  command  HDGLXXX  or  HDGRXXX,  where  "X"  represents 
a  digit,  should  be  avoided  and  arrows  substituted  for 
the  "L"  or  "R". 
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APPENDIX  A 

MESSAGE  FORMATS  AND  RAW  DATA 


All  reaction  times  in  seconds 

An  "E"  indicates  a  response  error,  and  reaction 
times  for  these  errors  were  not  recorded 

DM  *  Dot  Matrix  Characters;  ST  *  Stencil 
Type  Font;  SEG  *  16 -Segment  Characters 


A  blank  in  these  data  indicates  failure  of  a 
slide  to  drop  in*o  the  projector  properly  or 
a  failure  of  the  timer  to  reset. 
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TABLE  A- 2.  COMMANDS  REQUIRING  LEFT  TURN 


TABLE  A-4 .  COMMANDS  REQUIRING  DESCENT 


TABLE  A- 5.  COMMANDS  REQUIRING  RIGHT  TURN 


TABLE  A- 6.  COMMANDS  REQUIRING  ACKNOWLEDGEMENT 


APPENDIX  B 


SLIDE-PROJECTION  SEQUENCE 


APPENDIX  B-l 


SLIDE  PROJECTION  SEQUENCE 


Slide  Tray  1 


Seq. 

Slide 

Seq. 

Slide 

Sea- 

Slide 

1 

14 

25 

82 

49 

61 

2 

92 

26 

144 

50 

99 

3 

115 

27 

31 

51 

122 

4 

72 

28 

53 

52 

79 

5 

134 

29 

18 

53 

141 

6 

21 

30 

96 

54 

28 

7 

43 

31 

119 

55 

50 

8 

6 

32 

74 

56 

13 

9 

37 

33 

136 

57 

91 

10 

59 

34 

23 

58 

114 

11 

64 

35 

45 

59 

70 

12 

101 

36 

9 

60 

108 

13 

124 

37 

40 

61 

131 

14 

87 

38 

110 

62 

41 

15 

5 

39 

73 

63 

12 

16 

36 

40 

135 

64 

90 

17 

58 

41 

22 

65 

113 

18 

62 

42 

44 

66 

69 

19 

100 

43 

8 

67 

107 

20 

123 

44 

39 

68 

130 

21 

78 

45 

109 

69 

85 

22 

140 

46 

63 

70 

3 

23 

27 

47 

102 

71 

34 

24 

49 

48 

125 

72 

56 

Slide 

Tray  #1  presented  first  to 

subjects 

1  through  5. 

Slide 

Tray  #2  presented  first  to 

subjects 

6  through  10. 
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1 

75 

25 

7 

49 

68 

2 

137 

26 

38 

50 

106 

3 

24 

27 

60 

51 

129 

4 

46 

28 

65 

52 

86 

5 

11 

29 

103 

53 

4 

6 

89 

30 

126 

54 

35 

7 

112 

31 

81 

55 

57 

8 

67 

32 

143 

56 

20 

9 

105 

33 

30 

57 

98 

10 

128 

34 

52 

58 

121 

11 

84 

35 

16 

59 

77 

12 

2 

36 

94 

60 

137 

13 

33 

37 

17 

61 

26 

14 

55 

38 

80 

62 

48 

15 

66 

39 

142 

63 

19 

16 

104 

40 

29 

64 

97 

17 

127 

41 

51 

65 

120 

18 

83 

42 

15 

66 

76 

19 

1 

43 

93 

67 

138 

20 

32 

44 

116 

68 

25 

21 

54 

45 

71 

69 

47 

22 

117 

46 

133 

70 

10 

23 

95 

47 

132 

71 

88 

24 

118 

48 

42 

72 

111 

Slide 

Tray  #1  presented  first 

to  subjects 

1  through  5. 

Slide 

Tray  # 2 

presented  first 

to  subjects 

6  through  10 . 
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The  following  brief  appendix  is  provided  for  the  reader 
who  may  be  unfamiliar  with  or  requires  review  of  statistical 
methods  and  terminology. 

Statistically,  it  is  never  possible  to  prove  that  one  set  of 
measurements  is  different  from  those  obtained  in  measurement  of  a 
different  parameter  or  variable;  that  is,  the  numerical  comparison 
of  t  vo  populations.  Statistics,  instead  provide  means  for  de¬ 
termining  how  often  differences  measured  for  two  or  more  parameters 
would  occur  by  chance;  this  is  the  probability  value.  Thus  a 
probability  of  .01  indicates  that  by  chance  the  determined  dif¬ 
ferences  in  measurements  would  occur  only  one  time  in  a  hundred, 
and  for  a  probability  of  .001,  only  one  time  in  a  thousand. 

Calculations  of  such  probabilities  utilize  the  properties 
inherent  in  the  variability  of  measurements  of  data  points.  Three 
data  points  have  values  of  2,  2  and  2  have  a  mean  (average)  value 
of  2;  similarly,  three  data  points  having  values  of  1 ,  2  and  3 
also  have  a  mean  value  of  2,  but  here  the  values  vary  around 
this  mean.  "Variance"  is  the  measure  of  such  dispersion  of 
values,  and  is  based  upon  the  square  of  the  differences  between 
the  individual  measurements  and  the  mean  value. 

When  two  such  sets  of  measurements  having  different  means 
values  are  available,  a  t-test  permits  computation  of  the  pro¬ 
bability  that  these  differences  might  occur  by  chance.  Tables 
of  required  values  of  t  for  different  levels  of  statistical 
significance  are  available  in  any  standard  text  on  Statistics. 

The  technique  of  analysis  of  variance  extends  the  concept 
to  permit  the  simultaneous  comparison  of  variables  in  multi¬ 
dimensional  arrays.  Here,  because  of  the  different  computational 
procedure,  different  numerical  values  are  required  to  establish 
various  levels  of  confidence.  Appropriate  values  here  ? re  found 
in  tables  of  F-retio,  again  available  in  any  standard  text  on 
Statistics . 
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As  the  number  of  measurements  of  any  discrete  parameter  is 
increased,  we  obtain  increased  confidence  that  che  mean  of  the 
measurements  becomes  increasingly  closer  to  the  true  value  of  that 
parameter,  and  lower  values  are  required  in  tables  of  t  and  F 
for  a  given  level  of  statistical  significance.  "Degrees  of  free¬ 
dom"  is  the  term  used  to  indicate  the  number  of  measurements,  and 
is  defined  as  one  less  than  the  total  number  of  measurements 
being  evaluated.  Similarly,  in  computing  F-ratios,  the  concept 
of  degrees  of  freedom  is  used  to  indicate  the  number  of  levels  in 
each  dimension  of  the  experimental  design.  The  difference  between 
the  sum  of  the  degrees  of  freedom  taken  up  by  these  levels  and 
the  total  degrees  of  freedom  available  represents  the  degrees 
of  freedom  attributable  to  interaction  among  or  between  the 
variables.  The  variances  associated  with  these  degrees  of  free¬ 
dom  for  interaction  are  defined  as  the  "error  term",  and  this  is 
utilized  in  portions  of  the  computation  of  the  F-ratio. 
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The  questionnaire  administered  to  the  subjects  at  the  com¬ 
pletion  of  their  experimental  runs  elicited  the  following 
information : 

Question  1:  "Did  the  buzzer  on  the  WIDCOM  provide  sufficient 
information  that  a  new  command  had  appeared?" 

Answers:  8  Yes,  0  No. 

Question  2:  "Did  you  have  any  difficulty  in  identifying 
what  the  new  information  was?" 

Question  3:  "Did  you  consider  the  synthetic  speech  to  be 
sufficiently  intelligible?" 

Answers:  4  Yes,  4  No. 

Question  4:  "Did  the  intelligibility  improve  with  practice?" 
Answers:  0  No,  8  Yes;  2  qualified  with  "slightly". 

Question  5:  "Did  you  consider  the  combination  of  the  two 
better  than  either  WIDCOM  or  synthetic  speech  alone?" 
Answers:  6  Yes,  1  No,  1  Blank 

Question  6:  "If  only  one  device  were  available,  which  would 
you  prefer?" 

Answers:  6  WIDCOM,  1  Blank,  1  Question  Mark. 

Question  7a:  "In  an  emergency,  do  you  feel  that  you  would 
react  faster  to  one  device  than  to  the  other?" 

Answers:  6  Yes,  1  No,  1  Question  Mark. 

Question  7b:  "If  so,  which?" 

Answers:  7  WIDCOM,  1  Synthetic  Speech. 

Question  8:  "Do  you  have  any  suggestions  as  to  how  either 
device  might  be  improved,  or  what  improvements  are 
required?" 

Answers : 

SI:  "Synthesizer  intelligibility.  Add  frequency  selection 
to  WIDCOM.  I  feel  WIDCOM  with  buzzer  was  very  good." 

S2 :  "The  two  used  together  would  be  fine." 

S3:  No  comments. 
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S4 :  "WIDCOM  display  could  be  higher  (slightly).  WIDCOM 
buttons  pop  out  if  pushed  and  released  rapidly.  Synthetic 
speech  and  headset  combination  could  be  reason  for  not 
hearing  synthetic  speech  clearly.  WIDCOM  lost  display 
twice. " 

SS:  "Voice  Synthesizer:  (1)  intelligibility,  ^vocabu¬ 
lary,  c.f.,  "climb  to"  instead  of  "ascend  to."  WIDCOM: 
flash  information  and  only  have  one  response.  Buzzer 
terminal  areas,  pilots  like  to  maintain  a  mental  situa¬ 
tion  plot,  therefore  the  V/S  could  be  used  much  more 
easily  to  inform  all  required  A/C  of  situation  changes, 
particularly  holding  information." 

S6:  "Voice  synthesizer  could  be  more  intelligible  in 
some  cases.  Also  a  little  slow.  Both  devices  were 
quite  helpful.  The  WIDCOM  was  particularly  useful  by 
providing  a  constant  display  of  the  last  issued  commands. 
Trying  to  maintain  speed  to  5  knots  is  quite  difficult, 
at  least  with  relatively  little  experience.  The  GAT 
simulator  is  extremely  sensitive  to  all  controls.  This 
is  somewhat  difficult  to  adjust  to.  The  voice  synthesizer 
was  generally  easy  to  follow,  but  it  might  be  more  dif¬ 
ficult  when  multiple  aircraft  are  being  controlled." 

S7 :  "Emergency  commands  are  unusual  and  unexpected  and 
therefore  hard  to  recognize  on  the  synthesizer.  Some 
key  that  an  unusual  and  important  command  is  being  trans¬ 
mitted  is  needed." 

S8:  "Fewer  errors  in  WIDCOM  (difficulty  in  clearing  and 
obtaining  some  displays  seen  inherent  in  this  particular 
system).  *Add  to  WIDCOM  or  voice  more  aid  in  capturing 
ILS  signals.  Manual  flying  is  difficult  in  low  visability 


*Note:  Electrical  interference  duing  the  runs  caused  occasional 
erratic  behavior  of  the  WIDCOM). 
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ho  any  additional  info,  such  as  small  vectoring  to 
intercept  ILS  oould  be  very  valuable.  This  is  vastly 
better  than  monitoring  all  ATC  traffic." 
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